Virtual experience monitoring mechanism

ABSTRACT

A comprehensive solution is provided to transforming locations and retail spaces into high-traffic VR attractions that provide a VR experience blended with a real-world tactile experience. A modular stage and kit of stage accessories suitable for a wide variety of commercial venues contains all of the necessary equipment, infrastructure, technology and content to assemble and operate a tactile, onsite VR attraction. Utilizing a modular set of set design and physical props, the physical structure and layout of the installations are designed to be easily rearranged and adapted to new VR content, without requiring extensive construction or specialized expertise.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the following U.S. ProvisionalApplications, each of which is herein incorporated by reference in itsentirety.

SERIAL FILING NUMBER DATE TITLE 62618405 Jan. 17, 2018 VIRTUALEXPERIENCE (DVR.0108) MONITORING MECHANISM

This application is related to the following co-pending U.S. PatentApplications, each of which has a common assignee and common inventors.

SERIAL FILING NUMBER DATE TITLE 15783664 Oct. 13, 2017 MODULAR SOLUTIONFOR (DVR.0101) DELIVERING A VIRTUAL REALITY ATTRACTION 15828198 Nov. 30,2017 METHOD FOR GRID-BASED (DVR.0101-C1) VIRTUAL REALITY ATTRACTION15828257 Nov. 30, 2017 GRID-BASED VIRTUAL (DVR.0101-C2) REALITYATTRACTION SYSTEM 15828276 Nov. 30, 2017 SMART PROPS FOR GRID-(DVR.0101-C3) BASED VIRTUAL REALITY ATTRACTION 15828294 Nov. 30, 2017MULTIPLE PARTICIPANT (DVR.0101-C4) VIRTUAL REALITY ATTRACTION 15828307Nov. 30, 2017 GRID-BASED VIRTUAL (DVR.0101-C5) REALITY SYSTEM FORCOMMUNICATION WITH EXTERNAL AUDIENCE 16116034 Aug. 29, 2018 APPARATUSAND METHOD (DVR.0101-C6) FOR GRID-BASED VIRTUAL REALITY ATTRACTION15873523 Jan. 17, 2018 METHOD FOR AUGMENTING (DVR.0103) A VIRTUALREALITY EXPERIENCE 15873553 Jan. 17, 2018 METHOD FOR GRID-BASED(DVR.0104) VIRTUAL REALITY ATTRACTION SYSTEM 15873589 Jan. 17, 2018MODULAR PROPS FOR (DVR.0105) A GRID-BASED VIRTUAL REALITY ATTRACTIONCONTROL OF PHYSICAL (DVR.0106)      OBJECTS IN A VIRTUAL WORLD VIRTUALEXPERIENCE (DVR.0109)      CONTROL MECHANISM

BACKGROUND OF THE INVENTION Field of the Invention

This invention relates in general to the field of virtual realityattractions, and more particularly to virtual reality attractions thatblend physical elements with VR representations.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features, and advantages of the presentinvention will become better understood with regard to the followingdescription, and accompanying drawings where:

FIG. 1 illustrates one embodiment of a modular stage with a firstarrangement of stage accessories to augment the illusion of a first VRexperience;

FIG. 2 illustrates the modular stage of FIG. 1 with a second arrangementof stage accessories to augment the illusion of a second VR experience;

FIG. 3A illustrates the modular stage of FIG. 1 illustrating a labeledgrid of separable modular stage sections, each having a plurality of pegholes for fixing the stage accessories to the modular stage;

FIG. 3B is an enlarged view of a separable modular stage section,showing a labeled secondary grid of peg holes in the modular stagesection;

FIG. 4 is a perspective view of a wall equipped with pegs positionedover holes in a portion of the modular stage;

FIG. 5 illustrates a building facade accessory mounted on a modularstage;

FIG. 6 illustrates a VR representation of the building facade,embellished with an appearance of log siding and a tiled roof in awooded surrounding;

FIG. 7 illustrates a VR participant holding a flashlight prop whilepushing open a door of the building facade;

FIG. 8 illustrates a VR representation of an aged industrial doorway,with a flashlight-illuminated area that corresponds to the direction inwhich the flashlight prop is pointing;

FIG. 9 illustrates a VR participant walking over a wooden plank proppositioned on a modular stage platform;

FIG. 10 illustrates a corresponding VR representation of the woodenplank positioned over a deep gap separating two buildings;

FIG. 11 illustrates an elevator simulator on the modular stage;

FIG. 12 illustrates a corresponding VR representation of a VR elevator;

FIG. 13 illustrates a VR participant holding a firearm prop;

FIG. 14 illustrates a corresponding VR representation provided to the VRparticipant as he holds the firearm prop;

FIG. 15 is a block diagram illustrating one embodiment of a plurality ofdevices and systems used to augment or enhance a VR experience using agrid-aligned arrangement of props on a modular stage;

FIG. 16 is a block diagram illustrating one embodiment of a kit ofVR-represented “smart props” that are used to enhance a VR experience;

FIG. 17 is a block diagram illustrating components that are found invarious embodiments of a VR-represented object;

FIG. 18 is a functional illustration of the elevator simulator of FIG.11;

FIG. 19 illustrates one embodiment of a method of developing, deploying,and implementing a modular VR attraction;

FIG. 20 illustrates one embodiment of a method of building a stage forthe modular VR attraction;

FIG. 21 illustrates one embodiment of a method of assembling stageaccessories for the modular VR attraction;

FIG. 22 illustrates one embodiment of a method of digitallyinterconnecting the smart props with systems that service the VRexperience;

FIG. 23 illustrates one embodiment of a method of providing areal-world-integrated VR experience to participants;

FIG. 24 illustrates a method of providing a VR attraction to a pluralityof groups of VR participants in a staggered fashion;

FIG. 25 is a block diagram illustrating one embodiment of aninterconnected system of devices and systems used to augment or enhancea VR experience; and

FIG. 26 is a flow diagram illustrating a method for developing,deploying, and implementing a modular VR attraction that includesrecording, processing, and exploiting metrics associates with aparticipant's VR experience.

DETAILED DESCRIPTION

Exemplary and illustrative embodiments of the invention are describedbelow. In the interest of clarity, not all features of an actualimplementation are described in this specification, for those skilled inthe art will appreciate that in the development of any such actualembodiment, numerous implementation specific decisions are made toachieve specific goals, such as compliance with system-related andbusiness-related constraints, which vary from one implementation toanother. Furthermore, it will be appreciated that such a developmenteffort might be complex and time-consuming, but would nevertheless be aroutine undertaking for those of ordinary skill in the art having thebenefit of this disclosure. Various modifications to the preferredembodiment will be apparent to those skilled in the art, and the generalprinciples defined herein may be applied to other embodiments.Therefore, the present invention is not intended to be limited to theparticular embodiments shown and described herein, but is to be accordedthe widest scope consistent with the principles and novel featuresherein disclosed.

The present invention will now be described with reference to theattached figures. Various structures, systems, and devices areschematically depicted in the drawings for purposes of explanation onlyand so as to not obscure the present invention with details that arewell known to those skilled in the art. Nevertheless, the attacheddrawings are included to describe and explain illustrative examples ofthe present invention. The words and phrases used herein should beunderstood and interpreted to have a meaning consistent with theunderstanding of those words and phrases by those skilled in therelevant art. No special definition of a term or phrase (i.e., adefinition that is different from the ordinary and customary meaning asunderstood by those skilled in the art) is intended to be implied byconsistent usage of the term or phrase herein. To the extent that a termor phrase is intended to have a special meaning (i.e., a meaning otherthan that understood by skilled artisans) such a special definition willbe expressly set forth in the specification in a definitional mannerthat directly and unequivocally provides the special definition for theterm or phrase.

FIG. 1 illustrates one embodiment of a modular stage 1 with a first gridaligned arrangement 11 of stage accessories 14, 16, 18, 70, 110, 120 toaugment the illusion of a first VR experience/representation. The stageaccessories 14, 16, 18, 70, 110, 120 are provided as part of a VR stagekit 11. The stage accessories 14, 16, 18, 70, 110, 120 are assembled tothe stage 1 according a plurality of stage plans or arrangements thatcorrespond to a plurality of VR representations (aka “VR worlds”)provided in a VR attraction. The stage accessories 14, 16, 18, 70, 110,120 include set pieces and props. For example, FIG. 1 illustrates afacade 14 with a window 15 and door 6, a rock 18 attached to a perimeterwall 5, a flashlight prop 120 and a firearm prop 110 resting on a desk16, and a plank 70 on resting on a floor of a modular stage platform 3.The accessories 14, 16, 18, 70, 110, 120 give virtual realityparticipants sensory feedback that augments a virtual realityrepresentation. Some of the accessories 14, 16, 18, 70, 110, 120 maycomprise fittings 17 (such as pegs) to mount them to the modular stageplatform 3.

A modular stage 1 comprises a plurality of separable modular stagesections 2 designed to fit and cooperate with each other for ease ofassembly to form the stage 1. The modular stage 1 and its kit 11 ofstage accessories 14, 16, 18, 70, 110, 120 are configurable to fill adiscrete set of spatial areas-for example, 10 meters by 20 meters and 15meters by 15 meters-that might be found in a mall, theater, or otherretail space. Different spatial representations of a VR world arecreated to fit one or more of these areas and correspond to one or morestage plans or arrangements of accessories 14, 16, 18, 70, 110, 120 onthe stage 1.

In one embodiment, the modular stage 1 comprises a commerciallyavailable stage kit (not to be confused with the accessory kit 11described herein). Discretely positioned (and preferably regularlyspaced) accessory mounts 7 are either provided with, or incorporatedinto, the stage 1. In one embodiment, the stage 1 is elevated above theground, enabling signal lines 12 and power lines 13 to pass underneaththe platform 3 and through openings in the platform 3 (e.g., the pegholes 7) to service the accessories 14, 16, 18, 70, 110, 120 mounted onthe stage 1.

FIG. 3A illustrates a modular stage platform 3 made up of separablesquares or platform sections 2. For example, each square 2 may be 1 m×1m. FIG. 3B illustrates each square 2 as providing multiple aligned rowsof accessory mounts 7 in the form of holes that are spaced 1 decimeter(for example) apart from each nearest accessory mount 7. The squares 2are adapted to be connected to each other to create platforms 3 ofdifferent rectilinear dimensions. This enables the modular stage 1 tofit a wide range of conventional leasable commercial spaces.

The accessory mounts 7 are placed at preselected coordinates in agrid-like fashion in order to provide discrete places, readily andaccurately represented in a VR world, for the mounting of the stageaccessories 14, 16, 18, 70, 110, 120. In one practical embodiment, theaccessory mounts 7 are peg holes that are regularly spaced andconfigured for receiving accessories that have cooperating pegs. In thisapplication, the term “peg” is used in a broad sense to encompass largestructures as well as small structures. The peg holes 7 may be round,square, dimensioned to receive a dimensional board, or some other shape.The peg holes 7 are defined by a surrounding structure that, inconjunction with cooperating fittings or mounts 17 (e.g., pegs), providesufficient strength to fix and stabilize any mounted accessory 14, 16,18, 70, 110, 120. In an alternative embodiment, the stage platform 3 ismodified to incorporate pegs 17 for receiving accessories 14, 16, 18,70, 110, 120 with cooperating holes 7.

Any suitable substitute for a peg-and-hole system would also fall withinthe scope of the present invention, including mounts in the form ofseats, sockets, interconnectors, fasteners, couplers, couplings, clamps,hand-operated quick-release clasps, ties, pins, snaps, links, and thelike. The scope of the invention also includes any arrangement of femaleand male parts that attach one object to another, provided that theyfacilitate quick assembly and disassembly.

Collectively, the peg holes or other accessory mounts 7 of the modularstage platform 3 are aligned within rectilinear rows and columns,forming a grid or regular pattern 8. In one embodiment, the stage sideshave a primary set of alphanumeric markings 9, respectively, to identifyeach square 2 in the modular stage. In the 1 meter by 1 meter squareembodiment, this grid density provides a 1 meter by 1 meter level ofresolution. Each square or alternatively dimensioned platform section 2may also be labeled with its own secondary set of alphanumeric markings9, to identify each accessory mount 7 in the square or section 2. In the100-holes per square embodiment, this grid density provides a1-decimeter by 1-decimeter level of resolution. The invention is, ofcourse, not limited to these square dimensions or grid densities.

The assembly of the accessories 14, 16, 18, 70, 110, 120 to the modularstage platform 3 makes use of the positioning grid 8. For example, asnoted above, many of the accessories 14, 16, 18, 70, 110, 120 arearranged with fittings 17 (such as pegs) to mount them to the modularstage platform 3 at particular stage platform coordinates. The accessorymounts 7 cooperate with the fittings 17 to secure the accessories 14,16, 18, 70, 110, 120 to the platform 3. This aids in fast and accuratealignment with objects in virtual reality.

FIG. 4 illustrates this ease of assembly and disassembly by showing awall section 5 equipped with fittings 17 in the form of pegs positionedover peg holes 7 in a portion of the modular stage platform 3.Assembling the wall section 5 may be as simple as identifying thecorrect holes on the grid 8 using the alphanumeric markings 9 labelingthe grid 8, and inserting the pegs into the holes 7. Disassembling thewall section 5 may be as simple as lifting it from the stage 3.Quick-release clamps or connectors (e.g., clamps or connectors that donot require tools to operate) may optionally be employed, because theywould only modestly increase the amount of time needed to assemble anddisassemble the accessories 14, 16, 18, 70, 110, 120.

Parts may be added to or subtracted from the kit 11 to create newconfigurations. In one embodiment, the modular stage 1 includesperimeter walls 5 that are also covered in a labeled grid pattern 8,facilitating fastening of objects to the walls 5 in precise, discrete,exact, and vertically-aligned locations. A primary modular stageaccessory 5, such as an interior wall, may include its own labeled gridand pattern of accessory mounts (not shown) so that one or moresecondary modular stage accessories 14, 16, 18, 70, 110, 120 can beaccurately mounted to the primary stage accessory 5.

The grid-based approach described above is preferable to severalalternative approaches to aligning a virtual world with a physicalconstruction. One common alternative approach is to create a permanent“one-up” VR attraction that has not been designed in a modular fashion.It is not practical to update such attractions, limiting their abilityto bring in and appeal to repeat customers. Another approach wouldrequire that video sensors and/or other sensors be used to determine thelocation and orientation of each fixed, stationary modular stageaccessory 14, 16, 18. This approach in practice would provide a lessaccurate and/or reliable means of aligning the virtual and physicalworlds than this invention's approach, in which the objects of the VRrepresentation and the physical world are positioned at predeterminedcoordinates or grid points that select prepositioned accessory mounts 7.Another alternative would involve arranging accessories 14, 16, 18, 70,110, 120 on to the stage platform 3 at specified coordinates without thebenefit of a grid 8 or a patterned arrangement of peg holes or the like.A disadvantage of this approach is that it takes longer to assemble thestage, and with greater chance of error. Another disadvantage of thisapproach is that stage assemblers cannot assemble a stage as preciselyand quickly, this way, as they would with the grid-based approach. Theresult is that the physical and virtual worlds may not align asprecisely as they would with the grid-based approach.

As noted above, in one embodiment, the stage 1 is elevated above theground, enabling signal lines 12 and power lines 13 to pass underneaththe platform 3 and through openings in the platform 3 (e.g., the pegholes 7) to service the accessories 14, 16, 18, 70, 110, 120 mounted onthe stage 1.

FIG. 2 illustrates the modular stage 1 of FIG. 1 with a second stageplan or arrangement 19 of stage accessories 14, 16, 18, 70, 110, 120 toaugment the illusion of a second VR representation. FIGS. 1 and 2illustrate the speed and convenience with which accessories 14, 16, 18,70, 110, 120 can be accurately re-arranged on the stage 1 to correspondto different VR representations, with an ease that resembles rearrangingLego® blocks or placing one's ships at the start of a new Battleship®game. Advantageously, this makes it practical for proprietors to engagelocal customers with new experiences, keeping them coming back again andagain.

FIG. 5 illustrates a building facade 14 mounted on a modular stage. Thebuilding façade 14 comprises a door 6 and window 15 and has simple, flatdimensions. A 3D polystyrene rendering of a rock 18 has the contour of alarge rock or boulder and is coated with material like sand andsimulated moss to give it a rock-like tactile sensation. FIG. 6illustrates a VR representation 50 of the building facade 14,embellished with an appearance of log siding and a tiled roof in awooded surrounding.

FIG. 7 illustrates a VR participant 121 carrying a backpack 41 andwearing a VR headset 42. The backpack 41 carries a computer (not shown)running a VR engine. The VR participant 121 is holding a flashlight prop120 while pushing open the door 6 of the building facade 14. Theflashlight prop 120 comprises a conventional flashlight case. To createthe flashlight prop 120, any regular-sized battery, and optionally alsothe light bulb and lens, in the conventional flashlight case areremoved. These items are replaced with a smaller power source,orientation sensors and/or a self-tracking beacon so that a motiontracking system (not shown) can determine the location and orientationof the flashlight prop 120.

As shown in FIG. 8, a VR engine running on the computer in the backpack41 receives the ID, position, and location coordinates of the flashlightprop 120 and renders a VR representation 50 of a flashlight-illuminatedportion of the facade 14 and door 6, and a portion of an office beyondthe facade 14. In this VR representation 50, which contrasts with thewoodsy VR representation 50 of FIG. 6, the doorway is embellished tolook aged, with rust spots and paint chips. Elliptical areas 128 arerendered illuminated and the areas around the elliptical areas 128 arerendered dark, corresponding to the direction in which the flashlightprop 120 is pointing. This reinforces the illusion that the sensoryinformation received from the VR headset 42 is real.

FIG. 9 illustrates the VR participant 121 walking over the wooden plankprop 70 that is shown in FIG. 1 positioned on a modular stage platform3. The wooden plank prop 70 has a natural warp that causes it to wobblewhen crossed. The VR participant 121 walks very cautiously over theplank 70, even though the plank 70 is safely resting on the platform 3,and the VR participant 121 has a mere 1½ inches to fall should he losehis footing. The VR participant's fear is fueled by the VRrepresentation 50 depicted through the participant's headset 42. Asshown in FIG. 10, the VR participant 121 sees a virtual representation79 of the plank 70 precariously spanning a deep gap 78 separating twobuildings 76 and 77. And when the physical plank 70 wobbles, a motiontracking system (not shown) or accelerometer feedback wirelesslyprovided from the plank 70 detects the wobble. Using this data, the VRengine simulates the wobble and the disorienting effect of the wobble onin the VR representation 79 of the plank 70. Sound effects, such assqueaks, wood cracking and splintering further add to the illusion ofdanger.

FIG. 11 illustrates the VR participant 121 in one embodiment of anelevator simulator 80 comprising an enclosure 82 made of bars, thatchedplates, and/or gates. The simulator 80 may additionally comprise acontroller 85 such as a switch or buttons mounted to the enclosure 82.The elevator simulator 80 is substantially stationary, moving over aspan of only a few centimeters or inches to create an illusion ofascending or descending. FIG. 12 illustrates a VR representation 50 of acorresponding VR elevator 89. The VR elevator 89 is shown ascending ordescending one or more floors while the corresponding elevator simulator80 vibrates a platform (not shown) that is coupled to the enclosure 82.The elevator simulator 80 is further described in FIG. 18.

FIG. 13 illustrates the VR participant 121 holding and pointing afirearm prop 110. FIG. 14 illustrates a corresponding VR representation50 provided to the VR participant 121 as he holds, points, and shootsthe firearm prop 110. The VR representation 50 includes a depiction of aVR firearm 119 that is pointed in a direction that corresponds to thedirection in which the firearm prop 110 is pointed. The VRrepresentation 50 also depicts kill simulations 118 in response to theVR participant 121 “firing” the firearm 110.

FIG. 15 is a block diagram illustrating one embodiment of aninterconnected system 10 of devices and systems used to augment orenhance a VR experience using a grid-aligned arrangement 11, 19 of stageaccessories 14, 16, 18, 70, 110, 120 on a modular stage 1. The system 10comprises a plurality of sensors 20, a plurality of digital systems 30,a plurality of backpack-mounted local VR engines 40, and a plurality ofphysical “smart props” 60 that are simultaneously represented in acorresponding VR representation 50 experienced by VR participants 121.Some of the smart props 60 are interactive and designed for VRparticipants 121 to carry by hand as part of the VR experience. Somesmart props 60 have embedded prop actuators 45 to actuate mechanical,electrical, tactile, or heating elements incorporated into the smartprops 60.

The plurality of sensors 20 include an overhead sensor array 21,body-mounted tracking sensors 24 that are mounted on equipment (such asbackpacks 41 and/or headsets 42) carried by the VR participants 121, andfield-embedded sensors 27 embedded in one or more of the set pieces orprops 60 of the staged physical environment.

The plurality of digital systems 30 comprise motion tracking systems 31,a wire-based input-output system 38 (WireIO), and a merged realityengine 35. The motion tracking systems 31 detect the position andorientation of each VR participant 121 and each VR participant 121'shead and hands, as well as of smart props 60 used in the staged physicalenvironment. Suitable motion tracking technology already exists. Forexample, U.S. Pat. No. 8,638,989, issued Jan. 28, 2014, describestechnology that tracks a VR participant's hands in the virtual world sothat the VR experience can track the hands.

The motion tracking systems 31 send packets 33 of information—whichinclude the location coordinates, orientation, and uniquely identifyinginformation for each VR participant 121 or object—to the merged realityengine 35. The wire-based input-output system 38 (WireIO) is a networkof wired actuators 45 and sensors that are embedded in walls, doors,windows, smart props 60 and/or other objects of the physicalenvironment. Many of the wired sensors sense when a stage accessory 14,16, 18, 70, 110, 120, such as a door, a drawer, a door knob, or anelevator simulator platform, is opened or turned.

The merged reality engine 35 controls the actuators 45 to producemotion, direct a simulated breeze or wind (air), generate ambient heat,lock or unlock doors and windows, and/or generate other physicalphenomenon directed to and felt by the VR participant 121. The generatedmotions and other physical phenomena are accompanied by VR optics andsound that depict a VR representation 50 with a consistent surroundingand objects. Suitable technology for this purpose already exists.

For example, U.S. Patent Pub. No. 2016/0275722, published Sep. 22, 2016and incorporated herein by reference, describes systems and methods formerging a simulation experience with physical objects and sensorystimuli in a controlled physical environment.

The VR representation 50 includes VR representations of objects thatcorrespond—in apparent physical location and orientation—to the stagedphysical environment. As the VR participant 121 moves through the stagedphysical environment, the VR participant 121 is presented with a VRexperience that is consistent with, and that corresponds to, the VRparticipant 121's movement in the VR world. The VR world may be scaleddifferently than the physical world. For example, a wall that in thestaged physical world is 20 feet away may appear 40 feet away in the VRworld, but as the VR participant 121 approaches that wall, it appears tothe VR participant 121 that he/she is advancing faster as he/sheactually is. Although the scales differ, the physical world is perceivedby touch and other non-visual senses to be “consistent” and to“correspond” with the virtual world. Also, the texture of doorknobs,smart props 60, and other objects within the staged physical environmentneed not exactly match the textures that are displayed to the VRparticipant 121's eyes in the VR world. The existence of other sensoryfeedbacks may make a texture “feel” more like what the VR participant121 expects to feel than it otherwise would.

The merged reality engine 35 is configured with information to enhance aVR representation 50. The merged reality engine 35 receives andprocesses motion capture and other sensory feedback, and uses thatinformation to coordinate the physical “world” that exists on the stage1 with the VR representation 50 of that world. The merged reality engine35 tracks where each VR participant 121 is located and oriented withinthe staged physical environment, whether one of the VR participant 121'shands is reaching out to or holding a prop 60 in the staged physicalenvironment, and where the VR participant 121's head and/or eyes arepointing. The merged reality engine 35 provides physical coordination bycontrolling doors, windows, fans, heaters, simulated elevators, andother smart props 60 in the staged physical environment. The VRcoordination comprises sending signals regarding sensed conditions, VRparticipant 121 and prop locations, and actuator states to one or moreVR engines. For example, if a VR participant 121 moves a prop 60, thenthe merged reality engine 35 provides information to the VR engines 40to reposition and/or reorient the corresponding virtual props to matchthe participant-altered location and orientation of the physical props60.

In one embodiment, a single VR engine provides individualized VRexperiences to each of the VR participants 121. In what, with currenttechnology, is perhaps a more practical embodiment, separate VR engines40—each running on a backpack-mounted computer—are provided to each VRparticipant 121. The coordination of the physical and virtual worldsensures that the physical environment combines with the VR optics andaudio to provide a convincingly consistent sensory experience.

FIG. 16 illustrates a kit 11 of stage accessories that are used insidean exemplary staged physical environment to enhance a VR experience.Seven set pieces and smart props 60 are illustrated. Two of them—afirearm 110 and a flashlight 120—are designed for a VR participant 121to hold and carry by hand as part of the VR experience. Two more—adrawer 100 and a door 6—are designed to for a VR participant 121 to opento reveal a room or compartment. Another two—a plank 70 and an elevatorsimulator 80—are designed for a VR participant 121 to walk on and/oroperate. Also, a heating element 130 is designed to provide a source ofheat corresponding to a visual depiction of a heat source—such as aboiler—in the virtual world. Many more set pieces and smart props 60 arecontemplated, including, for example, light saber props, fencing gearprops, whips, tennis rackets, golf clubs, bats, balls, chair props, anda simulated moving platform. Further examples include land, sea, andair-based vehicular props. For example, a stationary rowing machine, inone embodiment, would enhance a VR simulation of rowing a boat across apond or channel. Also contemplated are a game controller or remote thatenables a VR participant 121 to alter or the VR scenery or switch the VRscenery between a plurality of VR representations 50 that correspond tothe staged physical environment. Of course, the selection andarrangement 10 of smart props 60 will be particular to the design of thestaged physical environment and the VR representation 50 it is createdto enhance.

The physical plank 70, which is also illustrated in FIGS. 1, 2 and 9,comprises a slightly warped wooden board. As illustrated in FIG. 10, theVR representation 50 visually and audibly simulates a corresponding VRplank 79. The physical plank 70 is embedded with a self-identifyingbeacon 61 (FIG. 17)—for example, LED lights powered by a battery packand controlled by a small programmable microcontroller 63 (FIG. 17) thatsets a sequence for illuminating the LED lights—that allows a motiontracking system 31 to track the position and orientation of the plank70. In one embodiment, the physical plank 70 is also embedded with asensor 66 (FIG. 17)—such as an accelerometer—to detect wobble in theboard. The microcontroller 63 wirelessly communicates—using, forexample, Wi-Fi or Bluetooth—the accelerometer output to the mergedreality engine 35, which is configured to generate a VR representation50 of a wobble in the corresponding VR plank 79.

The elevator simulator 80, which is partially illustrated in FIG. 11 andmore fully illustrated in FIG. 18, comprises a platform 81, an enclosure82 (such as bars or walls and a gate or a door), a plurality of springs83 (or pneumatic cylinders, pistons, or other actuators) supporting theplatform 81, and a vibration motor 86 to vibrate the springs 83 tocreate a sensation of elevator movement. Optionally, one or more sensors84 sense when VR participants 121 have entered or exited the simulator80. Controllers 85 such as a switch or buttons are provided forselecting a floor or elevator direction and/or causing the VR elevator89 to ascend or descend. Fans 87 project air on VR participants 121 toaugment a VR depiction of an open-air elevator with the sensation ofrapidly traveling through one. As illustrated in FIG. 12, thecorresponding VR representation 50 visually and audibly simulates acorresponding, but apparently fully-functioning, VR elevator 89. Whilein a typical embodiment, the actual appearance of the elevator simulator80 may be relatively simple and plain, the VR elevator 89 is depicted ina more visually engaging way.

The one or more sensors 84—e.g., a load sensor, a motion sensor, and/oran accelerometer—detect a person on the elevator simulator 80. As notedbefore, the springs 83 are vibrated by the vibration movement to createa sensation of movement. Feedback from testers indicates this to besufficient to create a convincing sensation. But more elaboratesimulators are also within the scope of the invention. In onealternative embodiment, for example, the springs 83 are pre-wound to apreset amount of compression or stretch from the springs' neutralpositions. To simulate an elevator ascent, the springs 83 are pre-woundinto the preset compressed position, and then suddenly released tosimulate the start of an elevator rising. To simulate an elevatordescent, the springs 83 are pre-wound into a preset stretched position,and then suddenly released to simulate the elevator lowering. Thevibration motor 86 simulates the travel of the elevator 80 from thestart position through the end of travel. To simulate the elevator 80coming to a stop, a cam (not shown) or other mechanical device is drivenby a second motor to cause the springs to compress in a simulateddescent or stretch in a simulated ascent followed by a second release,preferably to the position having the preset amount of compression orstretch to simulate the opposite movement of the elevator 80. Thissimulated “bounce” simulates the elevator 80 coming to a stop.

The merged reality engine 35 controls the elevator simulator 80. As theelevator simulator 80 simulates an ascent or descent and a stop, themerged reality engine 35 communicates information to thebackpack-mounted VR engines 40. To VR participants 121 for which the VRelevator 89 is in their field of view, the VR engines 40 simultaneouslypresent VR optics and auditory feedback depicting the corresponding VRelevator 89 traveling between one or more floors.

Advantageously, the elevator simulator 80 does not actually travelbetween floors. The platform 84 travels only a slight amountvertically—for example, less than 1 foot and even 4 inches or less—andyet the experience, when paired with the optics of the corresponding VRelevator 89, is genuinely realistic to nearly all, and certainly most,VR participants 121.

Also, advantageously, in one embodiment, the VR representation 50depicts two or more floors using a staged single-floor physicalenvironment. The elevator simulator 80 is used to “travel” between them.The same walls, doors, windows, smart props, and other objects in thestaged environment take on different appearances in each floor of the VRrepresentation 50. This enables a provider of the staged VR environmentto simulate an experience that appears much larger and more varied thanthe modular stage 1 and its assorted accessories themselves.

Returning to FIG. 16, the physical door 6 comprises a suitable door—suchas a conventional door—with a knob and hinges. The physical door 6includes rotary encoders 91 and 92 for the knob and one hinge,respectively. The rotary encoders 91 and 92, which may be analog ordigital, generate signals indicative of the angular position of the knoband door. The merged reality engine 35 uses the signals to generate a VRrepresentation 50 that visually and audibly simulates a corresponding VRdoor 99.

In one exemplary embodiment, the physical door 6 is an ordinary-lookingmetallic commercial door that is in good condition (few if any dents orscratches or paint chips) and lacks a frightening appearance. The hingesare lubricated so that they do not squeak when the door is opened orclosed. A door safety mechanism (not shown), powered by a spring,hydraulics, or other mechanism, creates resistance to the opening and/orclosing of the door 6. In one VR representation 50, such as shown inFIG. 8, the VR door 99 appears to be an industrial metal door covered inbadly chipped paint. A virtual danger warning sign hangs on the VR door99. As the door 6 is opened, the VR representation 50 presents squeakynoises as well as noises of the VR area behind the door 6. VR cobwebsare illustrated stretching and then breaking, an illustration that isfurther enhanced when the VR participant 121's face passes throughreal-world sticky or tacky threads or other filaments hanging down intothe staged physical environment. In a different VR representation 50,such as shown in FIG. 6, the VR door 99 appears to be a wooden door thatis part of a log cabin building.

The physical drawer 100 is part of a filing cabinet, desk, dresser, orother piece of furniture or equipment. In one embodiment, the drawer 100is equipped with a touch sensor 102 to detect touch, and a linearencoder 101 to detect the drawer position. In an alternative embodiment,a small door or lid, such as a safe door, is provided in place of or inaddition to the physical drawer 100. A handheld prop 60—such as afirearm prop 110—is preferably placed in the drawer, safe, or othercompartment, for a VR participant 121 to pick up and use.

The VR representation 50 visually and audibly simulates a correspondingVR drawer 109 or (in the alternative embodiment) door or lid, asillustrated, for example in the filing cabinet depicted in FIG. 8. If areal handheld smart prop 60 is in the drawer 100 or compartment, thenthe VR representation 50 simultaneously depicts a corresponding virtualhandheld prop.

Skipping briefly ahead, FIG. 17 illustrates some standard components ofsmart handheld props 60, many of which are incorporated into the firearmprop 110 and the flashlight prop 120 identified in FIG. 16. Thesestandard components include a tracking beacon 61, user controls 62, amicrocontroller 63, sensors 66, a transmitter 64, and haptic feedbackcomponents 67. It will be noted that the sensors 66 and/or transmitter64 may be incorporated into or on the microcontroller 63. In oneembodiment, the beacon 61 comprises self-identifying LEDs that are usedby a motion-tracking device, such as OptiTrack®, to determine theposition and orientation of the device 60. It will be understood thatany particular smart prop 60 may have fewer than all of the componentsillustrated in FIG. 17, as well as one or more components notillustrated in FIG. 17. Generally, the standard components of a smartprop 60 contribute to the delivery of a cohesive VR experience in whichthe physical prop 60 complements, rather than detracts from, the realismof the VR experience.

Returning back, FIG. 16 also illustrates a firearm prop 110. The firearmprop 110 is a handheld device with a gun-shaped form, including ahandle, a barrel-like section, a trigger or switch, and a cockingdevice. At least most of the standard components of FIG. 17 areincorporated into the firearm prop 110. The trigger of the firearm prop110, for example, would constitute a type of user control 62. In oneembodiment, the firearm prop 110 is equipped with a haptic feedbackcomponent 67 in the form of a trigger-activated spring recoil device 111to simulate the firing recoil of a real gun.

The firearm prop 110 may be an actual firearm that is unarmed andmodified for use in the staged physical environment. Preferably, thefirearm prop 110 replicates portions—e.g., substantially only theportions of a firearm that are typically gripped, pushed, pulled, ordepressed—of the form of any actual firearm, without including a chamberoperable to load real ammunition. For example, the firearm prop 110 maybe a lightweight polymeric replica of an actual firearm. The lightweightdesign makes it less susceptible to being used to actually harm—e.g., asa projectile or a battering device—another VR participant 121.

The VR representation 50 visually and audibly simulates the use of thefirearm prop 110. Whereas the firearm prop 110 is unable to actuallyfire ammunition, the VR representation 50 simulates the sound and lighteffects of ammunition being fired as the VR participant 121 depressesthe corresponding trigger—as well as the effects on the simulated livingand nonliving objects impacted by the simulated shots.

Advantageously, the motion tracking systems 31 detect the position andorientation of both the handheld smart props 60 (such as the firearmprop 110) and the position and orientation of the VR participant 121'shands. Moreover, touch sensors and accelerometers incorporated into thefirearm prop 110 enable the motion tracking systems 31 to detect whetherthe VR participant 121's hand is gripping and holding the firearm prop100. The merged reality engine 35 processes this information to enablethe VR engines 40 to accurately depict the VR depiction of the VRparticipant 121 as either not touching, merely touching or actuallygripping and holding the VR firearm 119. The merged reality engine 35also depicts a position and orientation of the corresponding VR firearm119 that matches the position and orientation of the firearm prop 110.

In other embodiments, not shown, physical smart props 60 for laser guns,light sabers, knives, swords, and hammers are also provided. Like thefirearm prop 110, they are equipped with components that enable them tobe simulated in the VR representation 50.

The handheld flashlight 120 preferably comprises a real polymericflashlight case or shell with a user-operable switch that has beenmodified with many of the standard components of a smart prop 60,including sensors 66, a microcontroller 63, and a transmitter 64 (whichmay be incorporated into the microcontroller 63). The sensors 66 detectwhether the flashlight prop 120 is being held, the position andorientation of the flashlight prop 120, and whether the flashlight prop120 is “switched” on or off. An actual flashlight bulb is unnecessary,because VR participants 121 would not see any actual light coming out ofthe flashlight 120, but rather the virtual depiction within the VRgoggles/headset 42 of the illumination created by a virtual flashlight129. Also, some of the space reserved in a conventional flashlight forbatteries is preferably used to house the sensors 66, microcontroller63, and transmitter 64.

The physical heating element 130 is a real-world device, such as a spaceheater, infrared heat lamps, or an oven (which may be connected to aduct having a microcontroller-operated exit door or valve). The VRrepresentation 50 illustrates a corresponding VR heat source 139, suchas a boiler, a raging fire, or a boiling pool of molten metal. Thephysical heating element 130 may stay on throughout the course of the VRexperience. Alternatively, the physical heating element 130 or valves orwindows or doors positioned to alternatively block or allow the releaseof heat are controlled to generate and provide sudden bursts of heat.Advantageously, the real heat felt by VR participants 121 dramaticallyenhances the sense of realism experienced by a VR participant 121.

As illustrated above, a smart prop 60 may be a replica or modifiedversion (or both) of a common utilitarian real-world object. Thesereinforce the typical expectations a VR participant 121 would have aboutthe corresponding VR object. However, there are many other types ofcontemplated smart props 60 that do not have the form or feel of acommon utilitarian real-world object. Such objects may reinforce theelements of suspense and surprise associated with the corresponding VRobject.

FIG. 19 illustrates one embodiment of a method of developing,implementing, and deploying a modular VR attraction. The methodcomprises a plurality of steps, some of which are re-orderable.

In step 205, using a commercially available VR development softwareprogram, create one or more VR representations 50 to which a physicalmodular stage 1 and its modular stage accessories can be fit, in aspatially realistic fashion, to enhance the illusion created by the VRrepresentations 50. In one embodiment, the modular stage and stageaccessories are pre-existing, and a previous VR representation 50 hasbeen created for use with the modular stage and stage accessories. In analternative embodiment, the modular stage and stage accessories arecreated after the VR representation 50 is created to augment and enhancethat VR representation 50 and its various spatial adaptations.

In step 210, select a space within a building or enclosure (e.g., amall, theater, mobile platform) for providing an integratedVR/real-world experience. In step 215, assemble the modular stage 1within the selected space. FIG. 20 elaborates on this step. In step 220,arrange a predefined set or kit 11 of set pieces and props 60 on thestage 1. More specifically, arrange the accessories according to a firstpreconfigured stage plan or arrangement 11 of location coordinates orreference points that correspond to a consistent arrangement ofvirtualized versions of those set pieces and props 60 in a first VRenvironment. Also connect power lines 13 and signal lines 12 toaccessories that need them. FIG. 21 further elaborates on this step. Instep 221, assemble set pieces (e.g., walls) and stationary props 60 todiscretely positioned accessory mounts 7. In step 222, position floatingprops 60 (e.g., plank, gun) at designated locations on the modular stage1 or in or on other accessories on the stage 1.

In step 225, load a VR server with motion capture, body tracking, andprop component control systems. Commercial packages are available thatmay be suitable, such as OptiTrack®. In step 230, interconnect VRserver, I/O network, overhead sensor array, and embedded sensors,actuators, and tracking devices of the integrated world and motiontracking systems. FIG. 22 further elaborates on this step. In step 235,install motion tracking software (e.g., OptiTrack Motive®) thataggregates and/or generates position and orientation info and thatidentifies hand positions.

In step 240, provide integrated VR/real-world experience to VRparticipants 121. FIG. 23 elaborates on this step.

In step 250, adapt the selected space for a new VR experience byreassembling structures, VR, props, and other physical objects withinthe modular enclosure according to another preconfigured stage plan orarrangement 19 corresponding to another VR representation 50.

It will be apparent to one of ordinary skill in the art that some of thesteps of FIG. 19 can be re-arranged without affecting the end-userexperience. Furthermore, some steps can be modified or omitted. Whethersuch rearrangements or modifications are within the scope of theinvention depends, of course, on the language of the claims.

FIG. 20 illustrates elements of a method 205 of assembling a stage 1 forthe modular VR attraction. In step 206, erect sensor scaffolding (e.g.,stage lighting stands & trusses). In step 207, mount video sensor arrayand other sensors on scaffolding. In step 208, assemble the modularplatform 3 and any perimeter walls 5.

FIG. 21 illustrates one embodiment of a method 220 of setting up stageaccessories for the modular VR attraction. In step 222, fix walls 5 andstationary props 60 to discretely positioned accessory mounts 7, forexample, by inserting fittings 17 such as pegs attached to the walls 5and props 60 into the accessory mounts 7, such as peg holes, of thestage platform. These accessories are designed to be restricted to afixed position on the stage platform 3, and are characterized within theVR world as being fixed. In step 222, position floating props 60 thatare not intended to be restricted to a fixed or stationary position onthe stage platform 3—for example, the plank 70 or the firearm 110—on thestage 1 at more generalized designated locations (e.g., specifying oneor more stage squares).

FIG. 22 elaborates on step 230 of FIG. 18. It illustrates one embodimentof a method of digitally interconnecting the smart props 60 with systemsthat service the VR experience. In step 231, connect the overhead sensorarray to the I/O network 20. In step 232, connect VR-world embeddedsensors, actuators and tracking devices to the I/O network 20. In step233, connect the VR server hosting the merged reality engine 35 to theI/O network 20 and the physical elements of the motion trackingsystem(s) 31.

FIG. 23 illustrates one embodiment of the step 240 of providing areal-world-integrated VR experience to VR participants 121. In step 241,equip VR participants 121 with VR goggles (e.g., Oculus Rift®) and VRengines (e.g., backpacks). In step 242, feed VR experience generated byVR engines to VR goggles as VR participants 121 walk through the stage1.

VR participants 121 may be advanced through the stage 1 in a pluralityof different fashions. In step 243, VR participants 121 are progressedthrough the stage 1 a single time in a sequential fashion. Inalternative step 244, VR participants 121 are advanced through at leasta portion of the stage 1 multiple times, while coupling differenttraverses of the same stage section with different VR representations50. This means that the objects of the multiply-traversed stage sectionsare associated with a plurality of VR representations 50, each onespecific to a different traverse.

In step 245, communicate position & orientation information from themotion tracking system(s) 31 to the VR server. In step 246, operateembedded set pieces and props 60 to produce motion, heat, sound, and/ortactile feedback that corresponds with VR effects illustrated for thecorresponding VR objects of the VR representation 50.

Various embodiments of the present invention use a computer system and acomputer network for executing one or more computer programs toassimilate sensory information, control actuable features—such asmotors, fans, heaters, and door locks—of stage accessories, and generatethe VR representations.

FIG. 24 illustrates a method 250 of providing a VR attraction to aplurality of VR participants 121 in a staggered fashion, staggered bygroup or individual. The method 250 is described herein with respect togroups, but would be equally applicable to an attraction in which oneparticipant is let in at a time. Moreover, the method 250 is notstrictly limited to the particular order in which the steps aredescribed. The VR attraction is subdivided into a plurality of connectedsections (aka rooms or booths) through which the groups or individualsare advanced, in a staggered, pipelined fashion.

In step 251, VR participants 121 are equipped with VR goggles andheadsets. In step 252, the VR participants 121 are equipped withbackpacks containing computerized VR engines, wherein each group isprovided with an instance of a VR world or game that starts when thegroup is admitted into the VR attraction, so that the instances aretemporally staggered between groups. In step 253, the VR participants121 are provided with objectives for VR participants 121 to complete ineach section. In step 254, a group of VR participants 121 is admittedinto a first section of the VR attraction. The group is given a minimumamount of time to complete the objective. If the group has not completedthe section objective in the minimum amount of time (block 255), and itis delaying the progression of newer groups from progressing forward(block 256), then in step 257, the objective or the VR representation 50is modified to help the group advance. For example, if an individual istoo frightened to cross the plank 70, then the VR representation 50 maymodify the appearance of the plank 70 as spanning two buildings into anappearance of a plank spanning a small brook. Or the VR representation50 is replaced with a true video representation of the stage section.Alternatively, an attraction operator may physically assist theparticipant through the section and into the next section. Flow returnsto block 255.

Once the group has completed the section objective (block 255), then aslong as the group has not yet completed the VR attraction (block 258),and provided the next section is vacant (block 260), then in step 261,the group is advanced to the next section. Once the group has completedthe VR attraction (block 258), then in step 259, the group is led out ofthe VR attraction. If the group has completed the section objective(block 255) but has not yet completed the VR attraction (block 258), andis blocked from advancing to the next section by another group (block260) then the VR attraction 50 will prolong the challenge or extend theobjective within the section.

Once a group vacates the first section of the VR attraction (block 261),then in step 254 another group is admitted into the first section of theVR attraction. Each group entering the first section begins a freshinstance of the VR world—and possibly even of a different VR world ifthe same arrangement 10 of stage accessories supports multiple VRrepresentations 50. In this manner, multiple groups can participate inthe VR attraction at the same time by having differently clockedinstances of the same (or another) VR world.

Advantageously, admitting individuals or groups into the VR attractionin a staggered fashion facilitates greater throughput. In some VRattractions, multiple players are admitted all at once. There, they playa single instance of a multiplayer game for durations extending from 15minutes to an hour. By contrast, the inventors have created a demo thathas an average duration of 5 minutes. That creates the possibility of 12game sessions/instances—each running for groups of as many as 6players—running per hour. If the average group size (includingsingle-member “groups”) is 3 players, then the VR attraction provides anefficient throughput of 36 players per hour. In short, the method ofFIG. 24 maximizes the space by running several instances of the game,facilitating a steady throughput of people.

In another advantageous embodiment, the system is configured todifferentiate between different numbers of group participants. Forexample, a group consisting of a single participant is presented with aVR representation of being solo or operating with artificialintelligence (AI) avatars. Also, or alternatively, the number of targetsor other objectives is reduced to a number suitable for a singleparticipant. But the greater the number of participants in the group,the greater the number of VR targets (e.g., monsters) to eliminate. Andeach participant is presented with a VR representation that includes aVR avatar of each other participant within the field of view. In oneembodiment, the VR headsets have microphones that pick up theparticipant's exclamations. These exclamations are incorporated into theVR representation 50 presented to other participants in the group.

In yet another enhancement, VR representations 50 being presented to oneor more participants within the VR attraction are simultaneouslylivestreamed to an operator console, a livestreaming website (such asTwitch® or YouTube®), or to terminals outside the VR attraction thatfriends and relatives can watch. The feature of livestreaming anindividual participant's experience to an operator enables the operatorto detect what might be slowing a group down, and aid the operator inassisting the group (block 257). Livestreaming the experiences to othersprovides an excellent marketing technique. In one embodiment, friendsand loved ones can watch the VR representation in a booth andcommunicate words of encouragement—or of warning—to the VR participants.These communications are transmitted into the VR participant's headset.

In yet a further embodiment, a smart prop 60 comprising a trackablecamera prop is provided. The camera prop enables participants tosimulate taking a picture within the VR attraction. A motion trackingsystem 31 tracks the position and orientation of the camera prop. Asensor, microcontroller, and transmitter combination in the camera proptransmits a signal indicating that a camera button has been pressed. Themerged reality engine 31 receives this signal and tracking informationto generate an image that is consistent with what a camera would havecaptured in the participant-held position and orientation were the VRrepresentation 50 a depiction of the real world. Participants are giventhe opportunity to purchase these simulated photographs that they tookwhile inside the VR attraction.

The present disclosure contemplates techniques that further enhance a VRparticipant's experience by providing mechanisms for recording,processing, and exploiting metrics corresponding to a VR participant'straversal through the VR attraction. These processed metrics, overlaidagainst the participant's VR experience, may be employed to generatesupplementary personalized VR products, provide data for subsequentanalysis, and affect real-time fixes for problems or issues arisingduring the VR participant's traversal.

Referring to FIG. 25, a block diagram is presented illustrating oneembodiment of an interconnected system 2500 of devices and systems usedto augment or enhance a VR experience using a grid-aligned arrangementof stage accessories on a modular stage. The system 2500 includessensors 20, one or more digital VR systems 30, one or morebackpack-mounted local VR engines 40, and physical smart props 60 thatare simultaneously represented in corresponding VR representationsexperienced by VR participants. Some of the smart props 60 areinteractive and designed for VR participants to carry by hand as part ofthe VR experience. Some smart props 60 have embedded prop actuators 45to actuate mechanical, electrical, tactile, olfactory, fluid, heating,or cooling elements which are incorporated into the smart props 60.

The sensors 20 include an overhead sensor array 21, body-mountedtracking sensors 24 that are mounted on equipment (such as backpacksand/or headsets) carried by the VR participants, and field-embeddedsensors 27 embedded in one or more of the set pieces or smart props 60of the staged physical environment.

The digital systems 30 include motion tracking systems 31, a wire-basedinput-output system 38 (WireIO), a merged reality engine 35, ananalytics recorder 351, one or more operator consoles 352, and one ormore product generators 353. The motion tracking systems 31 detect thelocation, orientation, and movement of each VR participant 121,including head, hands, and feet, as well as the location, orientation,and movement of smart props 60 used in the staged physical environment.

The motion tracking systems 31 send packets 33 of information—whichinclude the location coordinates, orientation, movement, and uniquelyidentifying information for each VR participant 121 or smart prop 60—tothe merged reality engine 35. The wire-based input-output system 38(WireIO) is a network of wired actuators 45 and sensors that areembedded in walls, doors, windows, smart props 60 and/or other objectsof the physical environment. Many of the wired sensors sense when astage accessory, such as a door, a drawer, a door knob, or an elevatorsimulator platform, is actuated (e.g., opened/closed, rotated, pushed,stepped on, etc.).

The merged reality engine 35 controls the actuators 45 to producemotion, direct a simulated breeze or wind (air), generate ambientcooling or heat, lock or unlock doors and windows, provide hapticfeedback (e.g., gun recoil), emit olfactory-stimulating substances(e.g., gunpowder, pizza, burning metal, etc.), emit mist, and/orgenerate other physical phenomenon directed to and sensed by the VRparticipants. The generated motions, feedback, substances, and physicalphenomena are accompanied by VR optics and sounds that depict VRrepresentations with consistent surroundings and virtual objects thatcomport with location, orientation, movement, and actuation ofcorresponding physical objects 60. Control of the actuators 45 may bedirected via the wire-based input-output system 38, or the mergedreality engine may employ wireless (e.g., Wi-Fi, IEEE 802.11-15, Zigbee,Bluetooth, etc.) links to directly communicate with embeddedcontrols/actuators 45 within the smart props 60 to sense actuation andto command feedback. For example, the merged reality engine maywirelessly receive actuation from the firearm prop 60 that its trigger45 has been actuated, and may wirelessly direct a recoil device 45within the firearm prop 60 to actuate in order to affect a morerealistic and believable VR experience.

The VR representations include virtual representations of objects thatcorrespond—in apparent physical location, orientation, movement, andactuation—to the staged physical environment. As the VR participantmoves through the staged physical environment, the VR participant ispresented with a VR experience that is consistent with, and thatcorresponds to, the VR participant's movement and smart prop's movementand actuation in the VR world. Some of the smart props 60 may includeactuators 45 that provide sensory feedback to, say, make an object feelmore like what the VR participant expects to feel than it otherwisewould.

The merged reality engine 35 is configured with information to enhance aVR representation 50. The merged reality engine 35 receives andprocesses motion capture and other sensory feedback, and uses thatinformation to coordinate the physical “world” that exists on the stagewith the VR representation of that world. The merged reality engine 35tracks where each VR participant 121 (along with hands and feet) islocated and oriented within the staged physical environment, whether oneof the VR participant 121's hands/feet is reaching out to or holding aprop 60 in the staged physical environment, and where the VR participant121's head and/or eyes are pointing. The merged reality engine 35provides timely physical coordination by sensing control actuations andby controlling doors, windows, fans, heaters, simulated elevators, andother smart props 60 in the staged physical environment. Thecoordination may include sending wired/wireless signals regarding sensedconditions, VR participant 121 and prop locations/orientations, andactuator states to one or more VR engines 40. For example, if a VRparticipant 121 moves a prop 60, then the merged reality engine 35provides information to the VR engines 40 to reposition and/or reorientthe corresponding virtual props to match the participant-alteredlocation and orientation of the physical props 60. Accordingly, thecoordination of the physical and virtual worlds ensures that thephysical environment combines with the VR optics and audio to provide aconvincingly consistent sensory experience.

As described above, the location, orientation, and movement of the VRparticipants are constantly tracked relative to and within agrid-aligned stage setting to provide corresponding timely and realisticVR representations in virtual space. The location, orientation, andmovement of the VR participants, along with location, orientation,movement, and (if applicable) actuation of controls 45 of the smartprops 60 are henceforth referred to as metrics corresponding totraversal of a VR participant through the VR attraction. Certain metricsmay create triggers in both the virtual space and the physical space.For example, as a VR participant approaches a virtual heat producingsource (e.g., virtual fire, virtual fireplace, a virtual stove, etc.), aphysical heater 45 (directly placed or embedded within a smart prop 60configured to correspond to the virtual heat producing source) may beactuated at a location and orientation that comports with that of thevirtual heat producing source to enhance the VR experience. Smart props60 such as fans may be employed to simulate wind, flying objects, motion(such as when the user is on a smart elevator device 60), to projectolfactory substances, and to emit moisture, etc. In the case of anelevator smart prop 60, the floor of the elevator 60 may include avibrational motor 45 and a fan 45 that together create the physicalimpression that the elevator 60 is moving up or down.

The merged reality engine 35 may correlate and overlay the metrics notedabove with a sequence of VR representations within the VR experiencesuch that the VR participant's unique experience within the VRattraction may be accurately simulated in both time and VR space. Themerged reality engine 35 may provide these overlaidmetrics/representations simultaneously to the analytics recorder 351 andto the one or more operator consoles 352. The analytics recorder 351records the streamed metrics/representations for exploitation subsequentto the VR participant's exit from the VR attraction. The merged realityengine 35, in concert with one or more operators at the one or moreoperator consoles 352, may detect and flag any issues or problems thatarise during the VR participant's traversal through the VR attraction,and may modify VR representations provided to the local VR engines 40 toaddress the issues/problems in real time. Likewise, the merged realityengine 35, in concert with one or more operators at the one or moreoperator consoles 352, may issue commands to embedded prop actuators 45to address the issues/problems in real time. In addition to addressingissues/problems, the overlaid metrics/representations may be employed bythe merged reality engine 35, in concert with one or more operators atthe one or more operator consoles 352 to enhance the VR participant'sexperience. Examples of modified VR representations and control ofembedded prop actuators 45 include, but are not limited to, opening andclosing doors, windows, and drawers; causing virtual transports (e.g.,elevators) to shut down; widening or narrowing virtual and/or physicalpathways; causing a virtual firearm to run out of ammunition; increasingolfactory emissions; etc.

Subsequent to the VR participant's exit from the VR attraction, therecorded overlaid metrics/representations may be analyzed by operatorsto improve performance of the VR attraction. The recorded overlaidmetrics/representations may also be provided to the one or more productgenerators 353 to develop personalized products for purchase by the VRparticipant. In one embodiment, the one or more product generators 353may comprise audio/video systems for generation of a DVD or downloadablevideo of the VR participant's experience within the VR attraction. Inanother embodiment, the one or more product generators 353 may comprisephotographic reproduction systems that may generate still photos of theVR participant's virtual experience. Further embodiments may comprise 3Dprinters, configured to products personalized artifacts obtained fromthe overlaid metrics/representations.

FIG. 26 illustrates one embodiment of a method 2600 of developing,implementing, and deploying a modular VR attraction. The method 2600comprises a plurality of steps, some of which are re-orderable.

In step 2605, using a commercially available VR development softwareprogram, create one or more VR representations to which a physicalmodular stage and its modular stage accessories can be fit, in aspatially realistic fashion, to enhance the illusion created by the VRrepresentations. In one embodiment, the modular stage and stageaccessories are pre-existing, and a previous VR representation has beencreated for use with the modular stage and stage accessories. In analternative embodiment, the modular stage and stage accessories arecreated after the VR representation is created to augment and enhancethat VR representation and its various spatial adaptations.

In step 2610, select a space within a building or enclosure (e.g., amall, theater, mobile platform) for providing an integratedVR/real-world experience.

In step 2615, assemble the modular stage within the selected space.

In step 2620, arrange a predefined set or kit of set pieces and props 60on the stage. More specifically, arrange the accessories according to afirst preconfigured stage plan or arrangement of location coordinates orreference points that correspond to a consistent arrangement ofvirtualized versions of those set pieces and props 60 in a first VRenvironment.

In step 2625, load a VR server 30 with motion capture system(s) 31, amerged reality engine 35, a WIREIO system 38, an analytics recorder 351,one or more operator consoles 352, and one or more product generators353.

In step 2630, interconnect VR server 30 with a network of sensors 20,local VR engines 40, and smart props 60.

In step 2640, provide integrated VR/real-world experience to VRparticipants while recording all motion capture and smart propinteraction metrics, and overlaying metrics with corresponding VRrepresentations. Simultaneously stream captured metrics and VRrepresentation to operator console.

In step 2660, employ the streamed metrics/representations to detect andflag problems or issues with the VR experience. Exemplary issues mayinclude opportunities to further enhance the participant's VRexperience.

In step 2670, via the one or more operator consoles, execute commands tocontrol the participant's virtual reality experience by modifying VRrepresentations and/or controlling one or more smart props.

In step 2680, subsequent to the participant's VR experience, utilize theone or more product generator s to generate one or more personalizedproducts based on the recorded overlaid metrics/representations.

A typical computer system (not shown) for use with the present inventionwill contain a computer, having a CPU, memory, hard disk, and variousinput and output devices. A display device, such as a monitor or digitaldisplay, may provide visual prompting and feedback to VR participants121 and a stage operator during presentation of a VR representation.Speakers or a pair of headphones or earbuds provide auditory promptingand feedback to the subject.

A computer network (not shown) for use with the present invention mayconnect multiple computers to a server and can be made via a local areanetwork (LAN), a wide area network (WAN), or via Ethernet connections,directly or through the Internet.

The particular embodiments disclosed above are illustrative only, andthose skilled in the art will appreciate that they can readily use thedisclosed conception and specific embodiments as a basis for designingor modifying other structures for carrying out the same purposes of thepresent invention, and that various changes, substitutions andalterations can be made herein without departing from the scope of theinvention as set forth by the appended claims.

What is claimed is:
 1. A grid-based virtual reality (VR) attractionsystem, comprising: a grid aligned stage kit that corresponds to aplurality of stage plans associated with plurality of VRrepresentations, said grid aligned stage kit comprising: smart props,comprising fixed smart props and moveable smart props, wherein one ormore of said smart props are simultaneously represented in one or moreof said plurality of VR representations experienced by a VR participant;a stage comprising a platform having a pattern of markings along atleast two dimensions, wherein said stage is configured with accessorymounts arranged on said platform, for affixing said fixed smart props tosaid platform, and wherein said pattern identifies coordinatescorresponding to a location of said accessory mounts; and a plurality ofstage sections that are configured to be interconnected according tosaid pattern to form said platform, wherein said platform isconfigurable according to said plurality of stage plans comprising atleast A×B and C×D, wherein A, B, C, and D are different from each other;a motion tracking system, configured to track identity, location,orientation, and movement of said smart props and said VR participant onsaid stage; a merged reality engine, coupled to said motion trackingsystem, configured to employ said identity, location, orientation, andmovement to generate said plurality of VR representations that simulatea virtual environment for said VR participant with virtually representedobjects corresponding to said identity, location, orientation, andmovement, and configured to overlay said identity, location,orientation, and movement with said plurality of VR representations; andan analytics recorder, coupled to said merged reality engine, configuredto record overlaid identity, location, orientation, movement, and VRrepresentations.
 2. The system as recited in claim 1, wherein saidmerged reality engine correlates and overlays overlay said identity,location, orientation, and movement with said plurality of VRrepresentations such that an experience of said VR participant may beaccurately simulated in both time and VR space.
 3. The system as recitedin claim 2, wherein said merged reality engine provides said overlaididentity, location, orientation, movement, VR representations to bothsaid analytics recorder and an operator console.
 4. The system asrecited in claim 3, wherein an operator at said operator console, inconcert with said merged reality engine, detects and flags problems thatarise during traversal of said VR participant through the VR attraction,and modifies VR representations to address said problems in real time.5. The system as recited in claim 4, wherein an operator at saidoperator console, in concert with said merged reality engine, issuescommands to one or more of said smart props to address said problems inreal time.
 6. The system as recited in claim 2, further comprising: aproduct generator, coupled to said analytics recorder, configured toemploy said overlaid identity, location, orientation, movement, and VRrepresentations to develop personalized products for purchase by said VRparticipant.
 7. The system as recited in claim 6, wherein said productgenerator comprises an audio/video system for generation of a DVD ordownloadable video of said experience.
 8. A grid-based virtual reality(VR) attraction system, comprising: a grid aligned stage kit thatcorresponds to a plurality of stage plans associated with plurality ofVR representations, said grid aligned stage kit comprising: smart props,comprising fixed smart props and moveable smart props, wherein one ormore of said smart props are simultaneously represented in one or moreof said plurality of VR representations experienced by a VR participant;a stage comprising a platform having a pattern of markings along atleast two dimensions, wherein said stage is configured with accessorymounts arranged on said platform, for affixing said fixed smart props tosaid platform, and wherein said pattern identifies coordinatescorresponding to a location of said accessory mounts; and a plurality ofstage sections that are configured to be interconnected according tosaid pattern to form said platform, wherein said platform isconfigurable according to said plurality of stage plans comprising atleast A×B and C×D, wherein A, B, C, and D are different from each other;a motion tracking system, configured to track identity, location,orientation, and movement of said smart props and said VR participant onsaid stage; a merged reality engine, coupled to said motion trackingsystem, configured to employ said identity, location, orientation, andmovement to generate said plurality of VR representations that simulatea virtual environment for said VR participant with virtually representedobjects corresponding to said identity, location, orientation, andmovement, and configured to overlay said identity, location,orientation, and movement with said plurality of VR representations; ananalytics recorder, coupled to said merged reality engine, configured torecord overlaid identity, location, orientation, movement, and VRrepresentations; and an operator console, coupled to said merged realityengine, configured to receive overlaid identity, location, orientation,movement, and VR representations.
 9. The system as recited in claim 81,wherein said merged reality engine correlates and overlays overlay saididentity, location, orientation, and movement with said plurality of VRrepresentations such that an experience of said VR participant may beaccurately simulated in both time and VR space.
 10. The system asrecited in claim 9, wherein said merged reality engine provides saidoverlaid identity, location, orientation, movement, VR representationsto both said analytics recorder and said operator console.
 11. Thesystem as recited in claim 10, wherein an operator at said operatorconsole, in concert with said merged reality engine, detects and flagsproblems that arise during traversal of said VR participant through theVR attraction, and modifies VR representations to address said problemsin real time.
 12. The system as recited in claim 11, wherein an operatorat said operator console, in concert with said merged reality engine,issues commands to one or more of said smart props to address saidproblems in real time.
 13. The system as recited in claim 9, furthercomprising: a product generator, coupled to said analytics recorder,configured to employ said overlaid identity, location, orientation,movement, and VR representations to develop personalized products forpurchase by said VR participant.
 14. The system as recited in claim 13,wherein said product generator comprises an audio/video system forgeneration of a DVD or downloadable video of said experience.
 15. Agrid-based virtual reality (VR) attraction method, comprising: providinga grid aligned stage kit that corresponds to a plurality of stage plansassociated with plurality of VR representations, the grid aligned stagekit comprising: smart props, comprising fixed smart props and moveablesmart props, wherein one or more of the smart props are simultaneouslyrepresented in one or more of the plurality of VR representationsexperienced by a VR participant; a stage comprising a platform having apattern of markings along at least two dimensions, wherein the stage isconfigured with accessory mounts arranged on the platform, for affixingthe fixed smart props to the platform, and wherein the patternidentifies coordinates corresponding to a location of the accessorymounts; and a plurality of stage sections that are configured to beinterconnected according to the pattern to form the platform, whereinthe platform is configurable according to the plurality of stage planscomprising at least A×B and C×D, wherein A, B, C, and D are differentfrom each other; via a motion tracking system, tracking identity,location, orientation, and movement of the smart props and the VRparticipant on the stage; via a merged reality engine, employing theidentity, location, orientation, and movement to generate the pluralityof VR representations that simulate a virtual environment for the VRparticipant with virtually represented objects corresponding to theidentity, location, orientation, and movement, and configured to overlaythe identity, location, orientation, and movement with the plurality ofVR representations; and via an analytics recorder, recording overlaididentity, location, orientation, movement, and VR representations. 16.The method as recited in claim 15, wherein the merged reality enginecorrelates and overlays overlay the identity, location, orientation, andmovement with the plurality of VR representations such that anexperience of the VR participant may be accurately simulated in bothtime and VR space.
 17. The method as recited in claim 16, wherein themerged reality engine provides the overlaid identity, location,orientation, movement, VR representations to both the analytics recorderand an operator console.
 18. The method as recited in claim 17, whereinan operator at the operator console, in concert with the merged realityengine, detects and flags problems that arise during traversal of the VRparticipant through the VR attraction, and modifies VR representationsto address the problems in real time.
 19. The method as recited in claim18, wherein an operator at the operator console, in concert with themerged reality engine, issues commands to one or more of the smart propsto address the problems in real time.
 20. The method as recited in claim16, further comprising: via a product generator, employing the overlaididentity, location, orientation, movement, and VR representations todevelop personalized products for purchase by the VR participant.